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SUMMARY 


In  the  early  1 980 ‘ s  AFHRL  undertook  a  research  project  to  model  enlisted 
retention  using  demographic  and  economic  predictors  and  incorporated  the 
results  in  a  computerized  decision  tool  used  by  Air  Force  personnel  planners 
and  decision  makers.  The  Air  Force  Retention  Analysis  Package  (AFRAP)  is 
discussed  in  an  AFHRL  technical  paper,  Air  Force  Retention  Analysis  Package: 
Users  Manual  (AFHRL-TR-89-1 0,  AD-A212  767).  The  underlying  econometric  model 
was  completed  in  1985,  using  reenlistment,  pay  and  policy  data  from  January 
1974  to  March  1982.  As  the  unusually  high  retention  patterns  of  the  early 
1980‘s  continued  into  the  mid-80's,  AFHRL  began  work  to  validate  the 
econometric  equations  using  reenlistment  data  from  April  1982  to  June  1986. 

The  econometric  model  was  reestimated  with  a  number  of  changes  including 
seven  new  variables;  recalculation  of  civilian  wage  as  a  function  of  age, 
sex,  race,  education  level  and  industry-wide  wages;  and  the  expansion  of  the 
data  base  to  include  reenlistment  decisions  made  since  March  1982.  Of  the 
new  variables,  three  accounted  for  seasonality  in  the  data,  one  reflected 
post-Vietnam  attitude  towards  the  military,  two  variables  accounted  for  prior 
knowledge  of  impending  changes  in  the  bonus,  and  one  modified  the  form  used 
to  model  changes  in  the  employment  rate.  These  enhancements  produced  a  more 
robust  model  of  reenlistment,  reducing  the  tendency  to  underestimate 
reenlistment  rates,  and  resulting  in  a  more  defensible  and  effective 
reenl istment  model . 


PREFACE 


The  validated  and  reestimated  econometric  model  of  Air  Force 
enlisted  retention  behavior  presented  in  this  paper  is  part  of  the 
Air  Force  Human  Resources  Laboratory  Force  Management  Program.  The 
econometric  model  underlies  the  Air  Force  Retention  Analysis  Package 
(AFRAP)  decision  tool  being  incorporated  in  Air  Force  and  OASD  force 
management  and  policy  analysis  systems.  This  enhanced  econometric 
model  and  resulting  AFRAP  software  will  provide  Air  Force  personnel 
policy  and  decision  makers  a  defensible  and  effective  management 
tool . 


The  authors  wish  to  thank  Dr.  Thomas  R.  Saving  and  Dr.  Guy 
Curry  for  technical  assistance  and  guidance  in  this  research 
project,  Mrs.  Kathy  Berry  and  her  staff  for  typing  and  editing  the 
manua1  and  Ms.  Rebecca  Wortman  for  her  review  of  this  paper. 
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I.  INTRODUCTION 


Since  1982,  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  has  been 
conducting  research  to  build  an  econometric  model  of  reenlistment  behavior.  This 
model  was  developed  and  tested  effectively  on  reeniistment  decisions  made  from 
1974  to  1982  (Saving,  Stone,  Looper  &  Taylor,  1985).  The  resulting  econometric 
equations  were  incorporated  into  the  Air  Force  Reenlistment  Analysis  Package 
(AFRAP),  an  interactive  software  routine  designed  to  permit  users  to  investigate  the 
effects  of  changes  in  economic  and  demographic  variables  on  predicted  reenlistment 
rates  in  specific  five-digit  Air  Force  Specialties  (AFSs).  In  addition,  AFRAP  was 
designed  to  project  the  impact  of  these  changes  on  the  force  distribution  by  year 
groups  in  each  AFS.  The  reenlistment  prediction  equations  were  developed  at  the 
five-digit  AFS  level  across  the  first,  second,  and  career  reenlistment  decision  points. 

The  technique  used  to  estimate  the  reenlistment  equations  was  a  maximum- 
likelihood  estimated  model,  called  probit,  which  has  a  binary  dependent  variable 
representing  an  airman's  choice:  1  if  the  airman  reenlisted  and  0  if  the  airman 
separated.  The  equations  estimated  the  probability  of  an  airman  reenlisting,  or  the 
rate  at  which  a  group  of  airmen  would  reenlist  given  a  set  of  demographic  and 
economic  conditions  relevant  at  the  time  of  the  decision.  The  conditions  captured 
in  the  explanatory  variables  included  military  compensation,  employment  rate, 
comparable  civilian  wage,  number  of  dependents,  marital  status,  education  level, 
mental  category,  race,  and  sex. 

Section  II  of  this  paper  presents  the  results  of  validating  the  reenlistment 
equations  from  Saving  et  al.  (1985)  against  more  recent  reenlistment  decisions. 
Several  statistics  were  calculated  to  determine  both  the  deviation  of  forecasted 
reenlistment  rates  from  their  actual  value  and  the  deviation  of  predicted  individual 
reenlistment  decisions  from  the  actual  decisions.  The  statistical  measures  used  for 
the  forecasting  validation  were  the  Coefficient  of  Multiple  Determination  referred  to 
as  the  Simulation  R^  (SR^),  Ex  Post  Root- Mean -Square  Forecast  Error  'RMSE), 
Theil’s  Inequality  Coefficient  (TIC)  and  its  three  components  (Bias,  Variance,  and 
Covariance),  Janus  Coefficient  (JC),  and  Percent  Successful  Predictions  (PSP). 

Based  on  the  evaluation  of  these  measures,  the  reenlistment  equations  were  re- 
estimated  to  improve  their  forecasting  capability.  The  new  equations  were  then 
validated  using  the  statistics  in  Section  II.  Section  III  presents  the  results  of  the 
re-estimation  and  validation.  Section  IV  summarizes  the  findings  and  presents 
recommendations  for  future  enlisted  force  retention  analysis. 


II.  VALIDATION  OF  THE  REENLISTMENT  EQUATIONS 


Several  different  measures  of  forecasting  credibility  are  presented  in  this 
section  to  analyze  how  accurately  the  reenlistment  equations  in  AFRAP  (Saving  et 
al.,  1985)  forecast  quarterly  reenlistment  rates.  Appendix  A  presents  a  definition 
and  brief  discussion  of  each  statistic.  The  statistics  were  used  to  identify 
forecasting  problems  encountered  in  the  simulations,  and  any  inconsistencies  among 
the  statistics  were  examined.  A  brief  discussion  of  the  data  will  follow  to  help 
clarify  some  of  the  problems  encountered  in  forecasting  reenlistment  rates. 


I 


The  reeniistment  equations  were  originally  estimated  using  data  on  individual 
airman  and  their  reenlistment  decisions  made  from  January  1974  to  March  1982 
(referred  to  as  the  original  sample).  The  validation  process  used  data  on  airmen 
making  reenlistment  decisions  from  April  1982  to  March  1986  (referred  to  as  new 
sample).  Calendar  quarters  were  used  as  the  basis  for  the  time  periods.  Equations 
for  all  three  categories  of  enlistment  were  validated:  115  AFS  equations  for  first 
term,  33  equations  for  second  term,  and  34  equations  for  career.  First  term  results 
are  the  most  meaningful  for  discussion  since  that  decision  point  is  the  crucial  one 
for  entry  into  the  career  force  and  little  variation  occurred  in  the  reenlistment 
rates  for  second  and  career  terms.  Hence,  the  focus  of  this  paper  will  be  on 
predicting  first-term  quarterly  reenlistment  rates. 


Background 

During  the  early  to  mid  1980's,  Air  Force  first  term  reeniistment  rates  reached 
historical  highs,  as  can  be  seen  in  Figure  I.  For  example,  the  reeniistment  rate  for 
fiscal  year  1982  (FY82)  was  56.7%  and  for  FY83  was  65.6%,  while  reeniistment  rates 
prior  to  FY81  were  rarely  over  40%.  FY81  marked  the  beginning  of  a  six-year  time 
period  (FY81  to  FY'86)  in  which  reeniistment  rates  averaged  56.5%. 

During  the  first  few  years  of  the  new  sample  period,  the  economic  environment 
of  the  private  sector  provided  limited  career  opportunities  and  uncertainty  for 
enlisted  personnel  who  were  at  the  first  decision-making  juncture  in  their  career. 
As  Figure  2  indicates,  the  new  sample  period  was  initially  one  of  high  unemployment 
for  the  20  to  24  year  old  male  civilian  labor  force,  limiting  the  number  of  career 
options  available  to  enlisted  decision  makers,  especially  first  termers.  By  the  end 
of  calender  year  1985,  the  20-to-24  year-old  male  unemployment  rate  had  fallen  to  a 
pre-1980  levels  of  10  to  12%.  Reeniistment  rates  had  not  adjusted  to  the  new 
employment  opportunities  which  were  arising  in  the  mid-1980’s. 

The  high  unemployment  rates  during  the  initial  part  of  new  sample  period,  the 
substantial  military  pay  raises  in  the  early  !980’s  (11.7%  in  October,  1980  and  14.3% 
in  October,  1981),  and  the  slow  adjustment  by  individual  reeniistment  decision¬ 
makers  to  an  improving  economy  were  key  reasons  for  the  improved  reeniistment 
rates  of  the  1980’s.  The  average  reeniistment  rate  for  the  original  sample  was 
approximately  39.9%  versus  59.1%  for  the  new  sample.  This  considerable  difference 
between  the  average  reeniistment  rates  for  the  original  versus  new  samples 
contributed  to  problems  in  forecasting  over  the  new  sample  period. 

The  differences  in  actual  reeniistment  rates  between  the  original  and  new 
samples  varied  among  the  AFSs.  For  example,  AFS  272x0  (Air  Traffic  Control) 
experienced  an  average  reeniistment  rate  of  48.7%  over  the  original  sample  period 
versus  55  4%  over  the  new  sample  period.  However,  the  difference  was  considerably 
larger  for  AFS  702x0  (Administration),  which  exhibited  rate<  of  46.4%  for  the 
original  sample  and  69.4%  for  (he  new  sample.  In  all  cases,  the  rates  for  the  new 
sample  were  higher  than  those  for  the  original  sample,  even  for  such  high-demand 
career  fields  as  computer  operators  and  computer  programmers,  which  had 
percentage  point  differences  of  8.8  and  12.5,  respectively. 


Unemployment  Rate 


In  addition  to  the  rising  reenlistment  rates,  the  Air  Force  also  experienced  a 
slight  increase  in  the  end-of-fiscal-year  force  levels.  The  force  level  increased 
approximately  2%  from  FY81  to  FY82  and  1%  from  FY82  to  FY83,  declining  to  a 
0.5%  increase  from  FY84  to  FY85.  This  modest  overall  rise  in  force  levels  of  2.5% 
paralleled  consistently  high  reenlistment  rates,  resulting  in  reductions  in  manning 
shortages  and  less  dependence  on  selective  reenlistment  bonuses. 


Bias 

As  previously  shown,  reenlistment  rates  for  all  categories  of  enlistment  were 
extremely  high  during  the  new  sample  period.  This  led  to  a  tendency  to 
underestimate  the  reenlistment  rates  for  the  new  sample  period.  The  mean  error 
(ME)  and  the  bias  statistic  from  TICs  (Theil,  1966)  were  used  to  determine  if  a 
persistent  bias  existed  in  the  new  sample  simulations.  The  ME  between  the  actual 
and  the  predicted  reenlistment  rates  indicates  whether  or  not  the  prediction  tends 
to  over-  or  underestimate  the  actual  rate.  The  bias  statistic  ranges  between  0.0 
and  1.0  and  indicates  whether  the  bias  is  persistent.  The  closer  the  ME  is  to  0.0 
the  less  likely  the  occurrence  of  a  persistent  bias. 

For  these  equations,  the  ME  averaged  0.129  with  a  standard  deviation  of  0.131. 
Only  13  of  the  115  equations  exhibited  a  tendency  for  overestimation.  Ten  of  these 
equations  were  considered  serious. bias  problems,  using  0.1  as  the  criteria  for 
identifying  a  consistent  bias  (Pindyck  &  Rubinfeld,  1981).  Of  the  102  equations  that 
exhibited  a  tendency  to  underestimate  the  reenlistment  rate,  92  of  the  equations 
were  considered  persistent  under  the  0.1  criteria.  Thus,  the  prediction  equations 
tended  to  persistently  underestimate  the  reenlistment  rates  for  the  April  1982  to 
March  1986  time  period.  This  is  not  surprising  given  the  historically  high 
reenlistment  rates  for  this  time  period.  Figure  3  presents  an  example  of  a 
persistent  prediction  bias  exhibited  by  AFS  431x1  which  typifies  the  bias  problems 
across  AFSs.  The  mean  quarterly  prediction  for  AFS  431x1  was  0.420  while  the 
actual  mean  for  the  new  sample  period  was  0.535.  The  prediction  was,  on  average, 
10  percentage  points  lower  than  the  actual  reenlistment  rate. 


Simulation 

Simulation  (SR^)  measures  how  well  the  estimation  equation  predicts 
compared  to  the  actual  sample  mean  and  has  a  value  between  minus  infinity  and  1.0. 
A  negative  number  indicates  that  the  prediction  error,  on  average,  is  larger  than 
the  difference  between  the  mean  and  actual  reenlistment  rate.  The  larger  the 
negative  number,  the  larger  the  error.  Of  course,  one  could  never  forecast  with 
the  sample  mean  because  one  does  not  know  the  sample  mean,  a  priori.  Given  the 
known  bias  in  the  data,  the  SR^  was  an  important  measure  of  the  forecasting 
credibility  of  the  estimation  equations. 


The  mean  value  for  SR^  across  115  first  term  equations  was  -0.941  with  a 
standard  deviation  of  2.665.  The  new  sample  SR^s  indicate  that  only  17  equations 
predicted  better  than  the  mean  reenlistment  rate,  i.e.,  they  exhibited  a  positive  SR^ 
value.  Using  SR^  as  a  guideline,  the  reeniistment  equations  which  performed  well 
still  tended  to  exhibit  an  underestimation  bias.  Twelve  of  these  equations  exhibited 
a  bias  using  the  0.1  criteria  for  the  bias  statistic. 
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Figure  3.  Actual  Versus  Predicted  Quarterly  Reenlistment  Rates  for  AFS  431x1  . 
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Thus,  SR2  exhibited  generally  unfavorable  results,  largely  due  to  the  significant 
differences  in  the  actual  mean  reenlistment  rates  between  the  original  and  new 
sample  periods.  Substituting  the  actual  mean  reenlistment  rate  from  the  original 
sample  for  the  actual  mean  reenlistment  rate  from  the  new  sample  in  the  calculation 
of  SR2  resulted  in  a  significant  improvement  The  revised  computation  using  the 
original  sample  rate  improved  the  new  sample  SR2s  from  a  mean  value  of  -0.941  to 
0.310.  Over  76%  of  the  equations  predicted  better  than  the  actual  mean  rate  for 
the  original  sample.  Of  the  98  reenlistment  equations  which  did  not  predict  better 
than  using  the  new  sample  mean  reenlistment  rates,  only  five  did  not  predict  better 
than  the  original  sample  mean  reenlistment  rate  was  used. 


Janua.,OuQtient 

The  poor  performance  of  the  equations  over  the  new  sample  as  shown  by  low 
SR2s  was  not  necessarily  a  reflection  of  a  poor  original  sample  prediction  equation. 
A  statistic  which  compares  the  level  of  forecasting  accuracy  between  new  and 
original  sample  predictions  is  the  Janus  Quotient  (JQ).  The  JQ  is  the  ratio  of  the 
sum  of  the  differences  between  actual  and  predicted  reenlistment  rates  in  the  new 
sample  to  the  sum  of  the  differences  between  actual  and  predicted  reenlistment 
rates  in  the  original  sample.  The  value  for  the  JQ  ranges  between  0.0  and  infinity 
with  a  value  of  1.0  or  less  indicating  that  the  prediction  of  the  equation  over  the 
original  sample  period  exhibits  the  same  or  lower  level  of  error  than  over  the  new 
sample  period.  Eleven  of  the  AFSs  had  a  JQ  value  of  below  1.0,  and  33  of  the  115 
AFSs  (30.4%)  had  a  JQ  value  of  less  than  2.0,  which  was  still  quite  acceptable 
(Koutsoyiannis,  1977).  Both  the  mean  and  median  JQ  values  were  high,  4.264  and 
3.141,  respectively.  If  the  eight  wont  AFSs  exhibiting  a  JQ  value  above  10  were 
excluded,  then  the  mean  JQ  value  improved  to  3.479. 


Root-Msan-Squarc  Simulation  Error 

The  Root-Mean-Square  Simulation  Error  (RMSE)  measures  the  deviation  of  the 
simulated  variable  from  its  actual  path  (Koutsoyiannis,  1977).  RMSE  ranges  between 
0.0  and  infinity  with  0.0  indicating  no  forecasting  error.  The  mean  value  for  the 
RMSE  was  0.217  with  a  standard  deviation  of  0.080.  The  median  value  for  RMSE 
was  0.199  exhibited  by  AFS  622x0  (Food  Services).  For  those  AFSs  exhibiting  a 
persistent  bias,  the  mean  RMSE  was  0.227  versus  a  mean  RMSE  of  0.140  for  those 
AFSs  without  a  persistent  bias. 


Theil’s  Inequality  Coefficient  and  Its  Components 

Theirs  Inequality  Coefficient  (TIC)  is  often  applied  to  new  sample  forecasts, 
and  it  can  also  be  reduced  to  its  three  components  to  provide  additional  insight 
into  determining  weaknesses  in  the  forecasting  models.  One  of  the  components,  the 
bias  statistic,  was  discussed  earlier.  TIC  ranges  in  value  from  0.0  to  1.0  with  0.0 
representing  a  perfect  forecast.  The  new  sample  values  for  TIC  averaged  0.228  with 
a  standard  deviation  of  0.125.  The  median  value  was  0.194  for  AFS  631x0  (Fuel) 
with  25%  of  the  equations  exhibiting  a  TIC  of  0.146  or  less. 
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TICs  computed  for  the  original  sample  averaged  0.1  SI,  30%  less  than  the  mean 
TIC  for  the  new  sample.  A  ranking  of  the  original  and  new  sample  TICs  (see  Table 
1)  provided  a  Spearman’s  coefficient  of  rank  correlation  of  0.288,  which  is 
statistically  significant  at  the  95%  level  of  confidence.  However,  a  low  TIC  value 
for  the  original  sample  did  not  necessarily  insure  a  low  TIC  value  for  the  new 
sample. 

The  variance  component  of  TIC  measures  the  ability  of  the  equations  to 
accurately  predict  the  degree  of  variability  in  the  reenlistment  rates.  The  variance 
ranges  between  0.0  and  1.0  with  0.0  representing  perfect  prediction  of  the 
variability.  The  variance  component  averaged  0.151  for  the  new  sample  with  a 
standard  deviation  of  0.138,  suggesting  that  although  some  of  the  equations  poorly 
forecasted  the  level  of  variability,  57  of  the  equations  had  a  variance  component  of 
less  than  0.1,  an  acceptable  cutoff  level  (Pindyck  Sc  Rubinfeld,  1981).  Large  upward 
or  downward  variations  in  the  quarterly  reenlistment  rates  did  occur  during  the  new 
sample  period,  often  resulting  in  large  prediction  errors.  The  variance  component 
indicated  that  the  large  upward  or  downward  variations  did  not  tend  to  dominate 
the  new  sample  forecasts. 

The  covariance  component  of  TIC  measures  the  degree  of  unsystematic  error. 
The  covariance  ranges  between  0.0  and  1.0  with  1.0  representing  the  best  covariance 
value.  The  covariance  is  more  likely  to  capture  the  unusually  large  variations  in 
the  reenlistment  rates.  The  covariance  component  averaged  0.333  with  a  standard 
deviation  of  0.246  for  the  new  sample  predictions.  This  low  value  represents  the 
difficulty  encountered  by  the  reenlistment  equations  in  predicting  the  historically 
high  reenlistment  rates  which  occurred  during  the  new  sample  period. 

Table  2  provides  a  comparison  of  the  validation  statistics.  Overall,  the 
statistics  reflect  the  general  inability  of  the  equations  to  predict  the  new  sample 
period.  The  minimum  value  for  RMSE  indicates  that  not  all  equations  encountered 
trouble  predicting  the  new  sample.  The  minimum  value  is  0.090,  and  the  value  one 
standard  deviation  below  the  mean  is  0.137.  The  mean  value  for  the  bias  statistic  is 
large  enough  such  that  one  standard  deviation  below  the  mean,  0.242,  is  still  at  an 
unacceptable  level  of  bias. 


Percent  Successful  Predictions 

The  percent  of  individual  airmen  which  the  equations  correctly  predicted  to 
either  reenlist  or  separate  averaged  60.5%  with  a  standard  deviation  of  6.1 
percentage  points.  The  equations  correctly  predicted  73.1%  of  the 
reenlistment/separation  decisions  over  the  original  sample  period.  The  percent 
successful  predictions  for  the  new  sample  period  ranged  as  high  as  79.9%  for  AFS 
751x2  (Training)  and  as  low  as  38.5%  for  AFS  672x1  (General  Accounting). 


Small  Sample  Sizes 

One  problem  that  was  prevalent  in  over  25%  of  the  first  term  career  fields  was 
the  small  number  of  reenlistment/separation  decisions  made  per  quarter.  The 
problems  caused  by  a  small  number  of  observations  is  that  large  swing  in  the 
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Table  1.  Comparison  of  the  Original  Versus  New  Sample  TIC*  Rankings 


Original 

sample 

AFSC 

New 

sample 

Original 

sample 

AFSC 

New 

sample 

1 

431x2 

30 

13 

751x2 

1 

2 

274x0 

2 

2 

274x0 

2 

3 

732x0 

49 

49 

741x1 

3 

4 

702x0 

17 

44 

304x4 

4 

5 

431x1 

14 

89 

472x2 

5 

6 

201x0 

24 

32 

602x1 

6 

7 

431x0 

41 

20 

423x0 

7 

8 

602x0 

22 

64 

423x3 

8 

9 

112x0 

42 

11 

423x5 

9 

10 

472x1 

38 

18 

272x0 

10 

11 

423x5 

9 

80 

427x4 

11 

12 

426x2 

86 

16 

651x0 

12 

13 

751x2 

1 

86 

871x0 

13 

14 

423x4 

65 

5 

431x1 

14 

15 

906x0 

92 

78 

304x1 

15 

16 

651x0 

12 

63 

391x0 

16 

17 

427x2 

63 

4 

702x0 

17 

18 

272x0 

10 

73 

231x1 

18 

19 

631x0 

60 

36 

908x0 

19 

20 

423x0 

7 

31 

271x1 

20 

21 

622x0 

56 

79 

427x1 

21 

22 

511x0 

70 

8 

602x0 

22 

23 

462x0 

64 

75 

362x4 

23 

24 

542x2 

43 

6 

201x0 

24 

25 

461x0 

93 

52 

271x2 

25 

26 

122x0 

34 

76 

326x4 

26 

27 

427x5 

53 

91 

566x1 

27 

28 

423x2 

37 

37 

276x0 

28 

29 

293x3 

61 

47 

552x0 

29 

30 

811x0 

77 

1 

431x2 

30 

‘Spearman’s  Rank  Order  Correlation  Coefficient  is  0.288  which  is 
statistically  significant  at  the  95%  level  of  confidence. 
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Table  2.  Comparision  of  Validation  Statistics 
Original  Equation 


Mean 

SD 

Minimum 

Maximum 

ME 

.129 

.131 

-.372 

.489 

Absolute  Mean 
Error 

.187 

.079 

.075 

.489 

RMSE 

.217 

.080 

.090 

.506 

SR2 

-.941 

2.665 

-9.999 

.929 

TIC 

.228 

.125 

.086 

.837 

Bias 

.516 

.274 

.000 

.948 

Variance 

.151 

.138 

.000 

.573 

Covariance 

.333 

.246 

.014 

.958 
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reenlistment  rates  were  attributable  more  to  the  small  numbers  of  decision  makers 
than  to  the  economic  or  demographic  variation  and  therefore  not  as  predictable. 

Across  the  115  first-term  reenlistment  equations,  the  average  number  of 
observations  per  quarter  ranged  from  a  low  of  9  to  a  high  of  520.  The  smallest 
25%  (in  terms  of  average  number  of  quarterly  observations)  averaged  only  17.8 
observations  per  quarter  compared  to  an  average  of  208.5  for  the  largest  25%  career 
fields.  Even  though  the  mean  reenlistment  rates  were  nearly  the  same  for  the 
smallest  and  largest  groups,  the  standard  deviations  of  the  those  rates  were 
significantly  different.  The  standard  deviation  for  the  smallest  group  was  0.180 
compared  to  a  standard  deviation  of  0.107  for  the  largest  group.  This  problem  was 
also  reflected  in  the  TIC  statistic.  The  smallest  group  had  an  average  new  sample 
TIC  of  0.294  versus  0.193  for  the  largest  group.  The  new  sample  TIC  averaged 
0.228  across  all  115  AFSs,  but  of  the  18  AFSs  with  an  average  of  less  than  20 
observations  per  quarter,  only  4  exhibited  a  new  sample  TIC  which  was  less  than 
0.2. 


Second  Term  and  Career  Validation  Results 


A  brief  look  at  second  and  career  term  results  reveals  that  they  were  not  as 
amenable  to  validation  as  first-term  equations  because  of  the  lack  of  variation  in 
reenlistment  rates  across  time  periods  for  second  term  and  career.  The  mean 
reenlistment  rates  for  second-term  AFSs  ranged  from  a  low  of  0.739  with  a  standard 
deviation  of  0.090  for  272x0  (Air  Traffic  Controllers)  to  a  high  of  0.901  with  a 
standard  deviation  of  0.045  for  995x0  (Recruiter).  The  career  term  exhibited  even 
less  variation  about  the  mean.  The  lowest  mean  reenlistment  rate  in  the  career 
term  was  0.960  with  a  standard  deviation  of  0.028,  exhibited  by  902x0  (Medical 
Service).  The  highest  mean  reenlistment  rate  in  the  career  term  was  0.997  with  a 
standard  deviation  of  0.008,  exhibited  by  114x0  (Aircraft  Loadmaster).  With  such 
minimal  variation  around  the  mean,  it  was  difficult  for  an  explanatory  equation  to 
improve  a  prediction  using  the  mean.  In  the  career  term,  six  equations  performed 
better  than  the  mean  reenlistment  rate  for  the  new  sample  as  indicated  by  the 
SR^s.  The  second  term  had  similar  results,  with  eight  equations  predicting  better 
than  the  mean  reenlistment  rate  for  the  new  sample. 


III.  RE- ESTIMATION  OF  THE  REENLISTMENT  EQUATIONS 

The  results  from  Section  II  can  be  summarized  by  the  following  three  problems 
identified  in  the  forecasting  of  first-term  reenlistment  rates  using  the  original 
Saving  et  al.  (1985)  equations  to  predict  reenlistment  decisions  made  in  the  new 
sample  period: 

1.  Significant  differences  between  the  new  and  original  sample  mean 
reenlistment  rates  contributed  to  a  persistent  bias  tendency  in  forecasting 
the  new  sample  reenlistment  decisions. 

2.  Reenlistment  rates  for  many  AFSs  exhibited  large  swings  which  were  not 
readiiy  explainable  by  the  reenlistment  equations. 


3. 


AFSs  with  a  small  number  of  transactions  per  time  period  exhibited 
significantly  higher  variations  in  reenlistment  rates  than  AFSs  with  large 
numbers  of  reenlistment  decisions.  Thus,  the  smaller  AFSs  were  more 
difficult  to  predict. 


Each  of  these  problems  was  resolved  to  some  degree  through  re-estimation  of  the 
original  reenlistment  equations  by  using  new  predictor  variables  and/or  additional 
time  periods. 

The  newly  estimated  equations  were  also  validated  to  determine  the  level  of 
overall  improvement  in  the  predictive  capability  of  the  reenlistment  equations  for 
the  new  sample  period,  as  well  as  the  entire  sample  period  (January  1975  through 
March  1986).  The  original  data  for  the  calendar  year  1974  were  found  to  be 
miscoded  for  reenlistments  and  thus  were  eliminated  from  the  sample.  Since 
:?ir>:Tcc!  variation  occurred  in  the  reenlistment  rates  for  second  term  and  career,  the 
focus  of  this  section  will  again  be  on  the  first- term  reenlistment  estimation 
equations. 


New  Reenlistment  Equation 

Several  new  variables  were  added  to  the  reenlistment  equations  to  enhance 
their  forecasting  credibility.  Seasonal  variables  were  included  for  the  fiscal 
quarters  of  the  year  to  account  for  large  swings  in  the  reenlistment  rates  due  to 
seasonal  variations,  e.g.,  end  of  fiscal  year  adjustments  in  the  fourth  quarter. 
Included  are  first  quarter  (FYQTR1,  October  through  December),  second  quarter 
(FYQTR2,  January  through  March),  and  fourth  quarter  (FYQTR4,  July  through 
September),  with  the  third  quarter  of  the  fiscal  year  (April  through  June)  implicit. 

The  calculation  of  civilian  wage  was  improved  by  using  age  earning  functions 
based  on  race,  sex,  academic  education  level,  and  industry.  The  new  calculation 
more  closely  approximates  the  civilian  earnings  opportunities  for  enlisted  personnel. 
An  age  earnings  function  was  estimated  for  each  of  12  demographic  cohorts  which 
were  based  on  sex  (male  or  female),  race  (Caucasian  or  non-Caucasian),  and 
education  level  (non-high  school  graduate,  high  school  graduate,  or  more  than  high 
school  graduate).  Each  of  the  estimated  functions  had  the  relationship  between 
earnings  and  age  as  shown  in  Figure  4. 

A  three-step  procedure  was  used  to  adjust  the  age-earnings  function  for 
industry  wages.  First,  using  an  average  age  for  the  civilian  population,  the 
appropriate  age-earnings  function  was  used  to  estimate  the  average  earnings  for  the 
cohort  in  which  the  individual  resided.  Secondly,  the  difference  between  the 
average  earnings  and  the  industry  wage  for  the  time  period  was  added  to  the 
intercept  to  produce  an  industry  adjusted  age-earnings  function.  Lastly,  the  age  of 
the  individual  was  used  to  estimate  his/her  expected  flow  of  civilian  earnings  for 
the  next  4  years  using  the  industry  adjusted  age  earnings  function  for  the 
calculation.  The  expected  flow  of  civilian  earnings  was  discounted  to  the  present 
using  a  10%  discount  rate  and  deflated  using  the  Consumer  Price  Index  (CPI). 

Two  additional  variables  were  added  to  the  estimation  equation  to  account  for  the 
change  in  reenlistment  behavior  due  to  anticipated  changes  in  the  Selective 
Reenlistment  Bonus  (SRB)  multiple.  For  example,  consider  what  might  happen  when 
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an  AFS  presently  paid  an  SRB  equal  to  a  multiple  of  2.0  experiences  a  reduction  in 
the  SRB  multiple  to  1.0  in  the  next  time  period.  Airmen  who  are  presently  eligible 
to  reeniist  within  the  next  12  months  or  less  and  already  intend  to  reenlist,  will 
reenlist  in  the  present  time  period  to  take  advantage  of  the  higher  SRB  multiple. 
Thus,  the  reenlistment  rate  for  the  present  time  period  will  experience  an  increase, 
while  the  reenlistment  rate  for  the  next  time  period  will  experience  a  decrease  due 
to  people  changing  the  timing  of  their  decision  to  reenlist.  The  coefficient  for 
BONUS  measures  the  change  in  the  reenlistment  rate  due  to  a  change  in  the  SRB 
multiple.  The  reenlistment  rate  would  be  expected  to  fall  when  the  SRB  multiple  is 
decreased,  but  unless  the  change  in  the  reenlistment  rate  due  to  the  timing  of 
decisions  is  accounted  for  in  the  estimation  of  the  BONUS  coefficient,  the  estimated 
coefficient  will  be  biased. 

To  eliminate  this  bias  in  the  BONUS  coefficient,  leading  and  lagging  variables 
were  included  for  those  time  periods  in  which  the  SRB  multiple  changed.  BFOR  was 
a  variable  equal  to  the  change  in  BONUS  from  the  present  to  the  next  time  period, 
while  BPAS  was  equal  to  the  change  in  BONUS  from  the  previous  to  the  present 
time  period.  For  those  time  periods  in  which  the  future  or  past  time  periods 
experienced  no  change  in  SRB  multiple,  the  values  of  BFOR  or  BPAS  were  zero. 
The  expected  relationship  between  reenlistment  and  BFOR  is  inverse,  i.e.,  increases 
(decreases)  in  the  SRB  multiple  in  the  next  time  period  cause  decreases  (increases) 
in  the  reenlistment  rate  in  the  present  time  period.  The  expected  relationship  for 
BPAS  is  direct,  i.e.,  increase  (decreases)  in  the  SRB  multiple  from  the  last  time 
period  to  the  present  time  period  cause  the  reenlistment  rate  in  the  present  time 
period  to  increase  (decrease). 

The  last  variable  added  to  the  reenlistment  equation  represented  the  change  in 
attitude  toward  military  service  since  the  end  of  the  Vietnam  War.  ATUD  is  an 
exponentially  declining  function  of  the  amount  of  time  which  has  lapsed  between 
the  date  of  the  decision  to  reenlist/separate  and  the  end  of  the  Vietnam  War 
(August  1972,  the  date  on  which  the  last  combat  troops  left  South  Vietnam).  ATUD 
is  expressed  as 


ATUD  -  (e-((t-c)/12))  *  1000 


where  e  is  the  base  of  the  natural  system  of  logarithms  (2.71828...),  t  is  the  time 
period  of  the  reenlistment/separation  decision,  and  c  is  the  date  identified  as  the 
end  of  the  Vietnam  War.  The  expected  relationship  for  ATUD  is  inverse  since 
reenlistment  rates  should  increase  the  farther  removed  the  decision  to 
reenlist/separate  is  from  the  Vietnam  War.  Also,  the  value  of  ATUD  declines  3S  t-c 
increases.  Figure  5  demonstrates  the  relationship  between  ATUD  and  time.  ATUD 
was  multiplied  by  1,000  to  minimize  the  problem  of  estimating  the  probit  model  with 
a  variable  which  declines  in  value  rapidly. 

The  final  modification  made  to  the  explanatory  variables  was  with  REMP.  In 
the  original  study  (Saving  et  al.,  1985),  REMP  exhibited  inconsistency,  with  33.3%  of 
the  equations  displaying  a  statistically  significant  negative  relationship,  but  with 
12.8%  showing  a  positive  significant  relationship.  The  square  of  REMP  was  added  to 
better  explain  the  relationship  between  reenlistment/separation  decisions.  The 
quadratic  form  is  shown  in  Figure  6.  The  expectation  is  that  REMP  is  inversely 
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Figure  5.  The  ATUD  Variable. 
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Predicted  Reenlistment  (Percent) 


related  to  reenlistment  over  the  range  of  reenlistment  rates  which  are  relevant  for 
the  combined  original  and  new  sample  time  periods. 


Probit  Estimation  Results 

In  general,  the  explanatory  variables  which  were  a  part  of  the  original 
estimation  equations  performed  approximately  the  same  as  in  the  original  equations 
(Saving  et  al.,  1985).  Table  3  presents  a  summary  of  the  first  term  results.  Due  to 
'he  remapping  of  several  AFSs,  the  number  of  estimated  equations  dropped  from  115 
to  113.  CWAGE  and  REMP  were  the  only  explanatory  variables  to  display  different 
results  than  the  original  equations.  The  proportion  of  equations  with  negative  and 
statistically  significant  CWAGE  declined  from  80.3%  to  47.8%.  Given  the  new 
methodology  for  calculating  CWAGE,  this  result  is  surprising.  The  extension  of  the 
estimation  time  period  may  have  contributed  to  this  change  in  CWAGE,  but  further 
analysis  is  required  to  determine  the  primary  causes  of  the  inconsistent  showing  of 
CWAGE. 

Since  REMP  has  an  hypothesized  inverse  relationship  with  reenlistment,  the 
downward  portion  of  the  curve  should  be  over  the  relevant  range  for  the  mean 
value  of  REMP.  To  determine  the  relationship  between  reenlistment  and  REMP, 
inflection  points  were  calculated  to  determine  whether  the  mean  value  for  REMP 
was  in  the  upward  or  downward  sloping  portion  of  the  curve.  REMP  and  REMP2 
were  both  statistically  significant  in  30  of  the  AFSs.  The  mean  value  for  REMP 
was  in  the  negative  portion  in  13  of  the  cases,  in  the  positive  portion  in  15  of  the 
cases,  and  was  at  the  inflection  point  in  2  cases.  As  with  the  previous  study 
(Saving  et  al.,  1985),  REMP  continued  to  exhibit  instability  at  the  disaggregated  AFS 
level  compared  to  the  aggregate  level  (DeVany  &  Saving,  1982). 

In  general,  the  new  variables  performed  as  expected.  ATUD  was  negative  and 
statistically  significant  in  92%  of  the  equations,  BFOR  was  negative  and  statistically 
significant  in  30%  of  the  equations,  and  BPAS  was  positive  and  statistically 
significant  in  51.9%  of  the  equations.  The  first  and  last  quarters  of  the  fiscal  year 
exhibited  the  most  consistent  influence  on  reenlistments.  FYQTR1  was  negative  and 
statistically  significant  in  40.7%  of  the  equations,  and  FYQTR4  was  negative  and 
statistically  significant  in  79.7%  of  the  equations.  FYQTR2  was  statistically 
significant  in  22%  of  the  equations,  15.9%  positive  and  6.19%  negative. 


Forecasting  Bias 

During  the  new  sample  period,  the  reenlistment  rates  for  all  categories  of 
enlistment  were  at  historical  highs.  This  led  to  the  finding  in  Section  II  that  the 
original  reenlistment  equations  tended  to  underestimate  the  reenlistment  rates  for 
the  new  sample  period. 

To  determine  if  a  persistent  bias  still  existed  in  the  new  sample  simulations 
post  re-estimation,  the  mean  error  (ME)  and  the  bias  statistic  from  TIC  were  used. 
The  ME  across  the  113  first-term  reenlistment  equations  averaged  0.037  with  a 
standard  deviation  of  approximately  0.044  over  the  new  sample  period  (refer  to 
Appendix  B  for  a  complete  listing).  The  original  115  equations  had  an  average  ME 
of  0.129.  Using  the  bias  statistic,  22.1%  (25  of  113)  of  all  equations  exhibited  a 
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Table  3.  Summary  Table  for  First  Term  AFS  Probit  Results* 


New  equation** 

Original  equation6 

Percent 

Percent 

Percent 

Percent 

negative 

positive 

negative 

positive 

ACED 

0.00 

32.74 

0.9 

41.0 

RACE 

79.44 

0.00 

96.6 

0.0 

AFQT 

11.11 

49.47 

1.7 

49.6 

DEPT 

93.81 

0.00 

90.6 

0.0 

MARST 

0.00 

78.85 

0.0 

86.3 

SEX 

31.86d 

4.42d 

45.4* 

2.8* 

BONUS 

72.70* 

6.48* 

75.6* 

1.2* 

BFOR 

17.10h 

30.30*1 

0.0 

0.0 

BPAS 

51.90* 

10.40* 

— 

— 

RMC 

78.76 

0.00 

70.9 

0.0 

TAFMS 

0.00 

100.00 

0.0 

96.6 

CWAGE 

7.96 

47.79 

0.9 

80.3 

REMP 

8.8S 

18.58 

12.8 

33.3 

REMP2 

18.58 

9.73 

- — 

— 

ATUD 

0.88 

92.04 

— 

— 

FYQTR1 

3.54 

40.71 

- — 

— 

FYQTR2 

15.93 

6.19 

— 

— 

FYQTR4 

0.00 

79.65 

— 

— 

‘Percentages  provided  for  those  coefficients  which  were  statistically  significant  at 
the  90%  level  of  confidence 

b  1 1 3  AFSs  with  sufficient  observations  for  analysis;  percent  of  113  AFSs  which  are 
significant 

'1 17  AFSs  had  sufficient  observations  for  analysis;  percent  of  117  AFSs 
^nly  107  AFSs  had  sufficient  numbers  of  females  for  analysis 
"Only  108  AFSs  had  sufficient  number  of  females  for  analysis 
*Only  77  AFSs  had  bonuses  (SRB) 

*Only  86  AFSs  had  bonuses  (SRB) 

hOnly  76  AFSs  had  sufficient  observations  for  BFOR 

•Only  77  AFSs  had  sufficient  observations  for  BPAS 
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tendency  for  overestimation.  Four  of  the  25  were  considered  a  serious  bias  problem 
using  0.1  as  the  criteria  for  identifying  a  consistent  bias.  Eighty-eight  of  the  re- 
estimated  equations  exhibited  a  tendency  to  underestimate  the  reenlistment  rate. 
Sixty-six  of  the  equations  were  considered  to  have  a  persistent  bias  using  the  0.1 
criteria.  The  original  equations  had  10  persistent  overestimators  and  92  persistent 
underestimators. 

Thus,  the  new  prediction  equations  still  exhibited  a  slight  tendency  for 
underestimation  of  the  reenlistment  rates  for  the  new  sample  period  but  improved  in 
their  prediction  capability.  The  magnitude  of  the  bias  problem  decreased  across 
AFSs.  The  underestimation  bias,  which  caused  problems  with  most  of  the  validation 
statistics  for  the  original  reenlistment  equations,  did  not  pose  as  serious  a  problem 
with  the  new  reenlistment  equations. 


Simulation  R2 


New  Sample  Period 


During  the  new  sample  period,  86  of  the  re-estimated  reenlistment  equations 
predicted  better  than  the  mean  reenlistment  rate,  i.e.,  SR2  was  greater  then  0.0. 
The  original  equations  predicted  better  than  the  mean  reenlistment  rate  in  only  17 
cases.  The  mean  SR2  across  113  first  term  reenlistment  equations  was  0.197  with  a 
standard  deviation  of  0.453.  This  compared  quite  favorably  with  the  mean  value  of 
-0.941  for  the  original  equation  projections.  The  maximum  value  for  SR2  was  0.849 
for  AFS  321x2  (Weapon  Control  Systems).  Forty-four  of  those  equations  predicting 
better  than  the  mean  reenlistment  rate  exhibited  a  bias  using  the  0.1  criteria. 


SR2  provided  better  results  for  the  entire  sample  than  for  the  new  sample. 
This  was  not  surprising  given  the  significant  difference  between  the  economic 
climates  of  the  new  and  the  original  sample  periods.  The  mean  reenlistment  rate 
for  the  original  sample  period  was  0.402  with  a  standard  deviation  of  0.094.  The 
mean  reenlistment  rate  for  the  new  sample  period  was  40.1%  higher,  with  an  average 
reenlistment  rate  of  0.564  and  a  standard  deviation  of  0.072. 


Entire  Sample  Period 

The  reenlistment  equations  also  were  evaluated  using  reenlistment  decisions 
made  during  the  entire  sample  period  which  provided  insight  as  to  the  predictive 
capability  of  the  new  equations  within  sample.  The  mean  SR2  for  the  entire  sample 
period  was  0.507  with  a  standard  deviation  of  0.139.  The  only  significant  difference 
between  the  equations  occurred  when  the  variable  BONUS  was  in  the  model.  The 
differences  in  the  predictive  capability  between  the  two  groups,  as  reflected  in  SR2, 
suggested  that  the  bonus  prediction  equations  tended  to  be  slightly  stronger.  The 
BONUS  group  had  a  20.7%  higher  SR2  than  the  non- BONUS  group.  The  mean  SR2 
for  the  77  equations  with  BONUS  during  the  entire  sample  period  was  0.537  with  a 
standard  deviation  of  0.128.  The  non-BONUS  equations  had  a  mean  SR2  of  0.444 
with  a  standard  deviation  of  0.140. 

The  rationale  for  the  superior  performance  of  the  BONUS  group  may  lie  in 
the  specification  of  the  equations  for  the  two  groups.  The  AFSs  in  the  BONUS 
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group  were  career  fields  experiencing  manning  shortages  at  various  points  in  time. 
To  alleviate  the  manning  shortages  caused  by  low  retention,  the  Air  Force  offers 
bonuses  to  induce  larger  numbers  of  reenlistments.  The  variation  over  time  in  the 
amount  of  bonus  offered  reflects  both  the  attempt  to  fine  tune  the  reenlistment 
rate  necessary  to  support  required  manning  levels  and  the  responsiveness  of 
personnel  within  career  fields  to  the  bonuses.  The  inclusion  of  the  BONUS  variable 
in  the  estimation  equation  for  the  BONUS  group  captured  the  systematic  variation 
in  the  reenlistment  rate  attributable  to  the  bonus. 

Career  fields  that  have  not  received  bonuses  to  induce  higher  reenlistment 
rates  have  either  experienced  balanced  personnel  manning  or  overages. 
Theoretically,  a  career  field  that  has  an  overage  should  receive  a  negative  bonus  to 
induce  voluntary  separations  or  retraining.  Since  a  negative  bonus  is  not  possible 
within  existing  Air  Force  compensation  policy,  no  directly  measurable  factor  is 
presently  available  for  properly  tracking  the  influence  of  a  negative  bonus  or  the 
personnel  policy  of  forced  retraining.  Thus,  the  random  component  in  the  variation 
of  the  reenlistment  rates  was  larger  in  the  non-BONUS  group. 

The  covariance  component  of  TIC  is  a  statistic  which  measures  this  level  of 
random  variation  not  captured  by  the  reenlistment  projections  of  the  equations. 
During  the  entire  sample  period,  the  covariance  for  the  BONUS  group  was  only  5.4% 
higher  than  that  of  the  non-BONUS  group,  but  during  the  high  retention  of  the  new 
sample  period,  the  covariance  for  the  BONUS  group  was  26.4%  higher  than  that  of 
the  non-BONUS  group.  This  is  not  surprising  since  the  high  retention  time  periods 
exhibited  more  tendencies  for  surplus  manning  rather  than  shortage  manning  within 
the  career  fields.  The  mean  covariance  were  0.875  and  0.830  for  BONUS  versus 
non-BONUS  AFS  groups,  respectively,  during  the  entire  sample  period 

The  BONUS  group  had  a  mean  covariance  of  0.700  versus  0.554  for  the  non- 
BONUS  group  during  the  new  sample  period.  The  percent  improvement  in  the 
covariance  during  the  new  sample  period  was  further  reinforced  by  the  fact  that  the 
median  value  for  the  BONUS  group  was  0.717  (AFS  915x0,  Medical  Material 
Specialist)  versus  0.529  for  the  non-BONUS  group  (AFS  551x0,  Pavement 
Maintenance  Specialist).  The  difference  in  covariance  statistics  between  the  BONUS 
and  non-BONUS  groups  further  emphasized  the  inability  of  the  reenlistment 
equations  to  account  for  the  effect  of  personnel  policies  to  correct  manning 
overages  within  non-BONUS  career  fields.  Given  the  high  reenlistment  rates  of  the 
new  sample  period,  one  would  expect  a  higher  level  of  internal  management  to  occur 
across  the  overage  AFSs,  and,  thus,  the  larger  difference  between  the  covariance 
values  for  the  BONUS  versus  non-BONUS  groups  over  new  sample  periods  gave 
evidence  of  this  fact. 


Root-Mean-Square  Simulation  Error 


New  Sample  Period 

For  the  new  sample  period,  the  RMSE  exhibited  a  mean  of  0.121  with  a 
standard  deviation  of  0.039.  This  was  44.2%  lower  than  the  0.217  mean  (standard 
deviation  of  0.080)  RMSE  exhibited  by  the  original  equations.  The  non-BONUS 
group  exhibited  only  a  5.1%  higher  RMSE  than  the  BONUS  group  but  a  44.0%  higher 
standard  deviation.  The  non-BONUS  group  had  a  mean  RMSE  of  0.125  with  a 
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standard  deviation  of  0.029  versus  the  BONUS  group’s  mean  RMSE  of  0.119  with  a 
standard  deviation  of  0.042.  The  difference  in  the  two  mean  RMSEs  was  smaller 
than  the  20.7%  difference  in  the  SR-  between  the  two  groups.  The  median  value  of 
RMSE  for  the  non-BONUS  group  was  0.116  for  AFS  571x0  (Fire  Protection)  versus  a 
BONUS  group  median  of  0.110  for  AFS  811x0  (Security  Police). 


Entire  Sample  Period 

The  RMSE  for  the  entire  sample  period  averaged  0.122  with  a  standard 
deviation  of  0.024.  Though  the  RMSEs  were  approximately  the  same  for  both 
periods,  the  59.3%  higher  standard  deviation  for  the  new  sample  reflected  the  effect 
of  the  difference  in  the  mean  reenlistment  rates  between  the  entire  and  the  new 
samples.  The  longer  estimation  period  tended  to  mitigate  the  high  and  low 
reenlistment  rates  when  the  equation  was  estimated.  Predictions  for  time  periods  in 
which  actual  reenlistment  rates  were  two  or  more  standard  deviations  from  the 
mean  tended  to  be  under*  or  overestimated  unless  a  significant  change  in  one  or 
more  of  the  explanatory  variables  precipitated  the  reenlistment  rate.  Only  a  slight 
difference  was  reflected  in  the  RMSE  between  the  non-BONUS  and  BONUS  AFS 
groups.  The  non-BONUS  group’s  RMSE  was  1.7%  higher  than  that  of  the  BONUS 
group  over  the  entire  sample  period.  The  mean  for  the  non-BONUS  group  was  0.123 
with  a  standard  deviation  of  0.022  versus  a  mean  RMSE  for  the  BONUS  group  of 
0.121  with  a  standard  deviation  of  0.025. 


Theil’s  Inequality  Coefficient  and  Its  Components 

New  Sample  Period 

The  TIC  values  over  the  new  sample  period  averaged  0.110  with  a  standard 
deviation  of  0.044.  This  indicated  that  most  of  the  new  reenlistment  equations  were 
predicting  considerably  better  than  the  original  equations,  which  had  a  mean  TIC  of 
0.228  with  a  standard  deviation  of  0.125.  Thus,  on  average,  the  new  equations  had 
a  49%  improvement  with  respect  to  the  TIC  statistic.  The  best  value  exhibited  by 
an  AFS  for  TIC  was  0.048  for  AFS  427x5  (Airframe  Repair),  a  very  favorable  result. 
The  median  value  was  0.104  for  AFS  431x1  (Tactical  Aircraft  Maintenance),  with 
46.9%  of  the  equations  exhibiting  TIC  of  0.1  or  less. 

The  variance  component  averaged  0.155  for  the  new  sample  with  a  standard 
deviation  of  0.128.  Both  new  and  original  equations  reflected  similar  abilities  in 
forecasting  the  level  of  variability  in  the  reenlistment  rates.  The  mean  of  the 
variance  component  for  the  original  equations  was  0.151  with  a  standard  deviation 
of  0.138.  Approximately  44%  of  the  first- term  reenlistment  equations  predicted  the 
new  sample  period  with  a  variance  component  of  less  than  0.1.  As  discussed 
previously,  large  upward  or  downward  variations  in  the  quarterly  reenlistment  rates 
did  occur  during  the  new  sample  period,  often  resulting  in  large  prediction  errors 
which  weakened  the  values  for  SR^.  The  variance  component  indicated  that  the 
large  upward  or  downward  variations  in  the  reenlistment  rates  did  not  dominate  the 
predictions  in  the  new  sample. 

For  the  new  sample,  the  covariance  component  averaged  0.654  with  a 
standard  deviation  of  0.185.  The  original  equations  exhibited  a  mean  value  for 
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covariance  of  0.333  with  a  standard  deviation  of  0.246.  The  mean  value  for 
covariance  improved  approximately  96%  with  the  new  equations. 


Entire  Sample  Period 

The  TICs  for  the  entire  sample  period  averaged  0.128  with  a  standard 
deviation  of  0.033.  The  best  value  of  TIC  was  0.066,  exhibited  by  AFS  431x2 
(Strategic  Aircraft  Maintenance),  while  the  worst  value  was  0.285,  exhibited  by  AFS 
316x1  (Instrumentation  Mechanic).  The  median  TIC  for  the  entire  sample  was  0.125 
for  AFS  982x0  (Medical  Services  Specialist).  Approximately  16.8%  of  the  AFSs 
exhibited  a  TIC  of  less  than  0.1.  The  BONUS  group  exhibited  a  mean  TIC  of  0.130 
with  a  standard  deviation  of  0.036  versus  the  non- BONUS  group  mean  of  0.126  with 
a  standard  deviation  of  0.024.  Though  this  is  the  reverse  order  of  RMSE  and  SR^, 
the  mean  TIC  values  are  nearly  the  same  with  the  BONUS  group  exhibiting  a  higher 
standard  deviation. 

The  variance  component  for  the  entire  sample  period  exhibited  a  mean  value 
of  0.123  with  a  standard  deviation  of  0.093.  The  bias  statistic  averaged  0.016  with 
a  standard  deviation  of  0.023.  The  covariance  averaged  0.861  with  a  standard 
deviation  of  0.105.  This  average  value  for  the  covariance  component  increased 
slightly  for  the  BONUS  group  to  0.875  with  a  standard  deviation  of  0.108.  The 
mean  covariance  for  the  non-BONUS  group  was  0.830  with  a  standard  deviation  of 
0.090.  The  best  covariance  value  for  the  entire  sample  period  was  0.993  exhibited 
by  AFS  423x4  (Aircraft  Pneudraulic  Systems  Mechanic),  a  member  of  the  BONUS 
group.  Approximately  41%  of  the  AFSs  exhibited  a  covariance  of  0.9  or  better. 

Fiscal  Year  FcrscastiPK 

Another  issue  of  concern  for  the  reenlistment  equations  was  their  ability  to 
predict  reenlistment  rates  by  fiscal  year.  Reenlistment  equations  may  predict 
quarterly  fluctuations  well,  yet  perform  poorly  at  the  fiscal  year  level.  In  general, 
the  more  aggregated  the  time  period  to  be  predicted,  e.g.,  annual  or  fiscal  year 
versus  quarterly,  the  better  the  new  equations  predicted.  The  fiscal  year  based  TIC 
calculated  for  the  entire  sample  period  averaged  0.077  with  a  standard  deviation  of 
0.029.  The  TIC  improved,  on  average,  39.8%  when  predicting  fiscal  year  versus 
quarterly  reenlistment  rates.  In  addition,  the  TIC  for  each  of  the  AFSs  improved 
when  comparing  fiscal  year  versus  quarterly  predictions.  The  largest  change  was 
54.6%  by  AFS  233x0  (Imagery  Production),  and  the  least  change  was  3.8%  by  AFS 
27 1  x2  (Operations  Resources  Management  Specialist).  On  average,  SR^  improved  34% 
from  0.507  to  0.680  from  quarterly  to  fiscal  year  predictions.  The  mean  RMSE 
improved  41.1%,  from  0.122  for  the  quarterly  predictions  to  0.072  for  the  fiscal  year 
predictions. 


Comparison  of  Original  and  New  Equation  Forecasting 

Table  4  presents  a  comparison  between  the  validation  statistics  of  the 
original  and  new  equations,  columns  1  and  2.  In  addition.  Table  4  provides  the 
validation  statistics  for  the  entire  sample  period  and  the  fiscal  year  using  the  new 
equations.  Comparing  columns  1  and  2,  the  validation  statistics  for  the  new 
equations  were  better  in  all  cases  than  those  for  the  original  equations.  This 
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Table  4.  Comparison  of  Validation  Statistics 
(Original  Equations  vs.  New  Equations) 


New  eauation 

New  sample 

New  sample  Entire  sample 

FY 

ME 

0.129 

0.037 

0.003 

0.001 

(0.131) 

(0.044)* 

(0.016) 

(0.017) 

Absolute  Mean 

0.187 

0.098 

0.096 

0.057 

Error 

(0.079) 

(0.030) 

(0.018) 

(0.015) 

RMSE 

0.217 

0.121 

0.122 

0.072 

(0.080) 

(0.039) 

(0.024) 

(0.020) 

SR2 

0.041 

0.306 

0.507 

0.680 

(.129) 

(0.257) 

(0.139) 

(0.177) 

TIC 

0.228 

0.110 

0.128 

0.077 

(0.125) 

(0.044) 

(0.033) 

(0.029) 

Bias 

0.516 

0.191 

0.016 

0.034 

(0.274) 

(0.181) 

(0.023) 

(0.056) 

Variance 

0.151 

0.155 

0.123 

0.227 

(0.138) 

(0.128) 

(0.093) 

(0.197) 

Covariance 

0.333 

0.654 

0.861 

0.734 

(0.246) 

(.185) 

(0.105) 

(0.214) 

"Standard  Deviation. 
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indicates  the  overall  improvement  in  the  predictive  capability  of  the  new  equations 
during  the  new  sample  period.  Of  course,  it  should  be  noted  that  the  predictions 
for  the  new  sample  period  using  the  original  equations  are  out-of-sample  predictions 
whereas  the  new  sample  predictions  using  the  new  equations  are  in-sample.  An 
additional  test  would  be  to  select  a  time  period  which  is  out-of-sample  for  both 
sets  of  equations.  However,  these  data  were  not  available  for  study  in  this  effort 

One  interesting  result  from  the  comparison  of  the  new  equation  predictions 
between  the  entire  sample  and  the  new  sample  was  the  large  difference  between  the 
mean  values  for  ME,  0.037  and  0.003  for  the  new  and  entire  samples,  respectively. 

The  large  difference  in  the  ME  values  was  not  reflected  in  the  means  of  the 
absolute  value  of  the  MEs,  0.098  and  0.096  for  the  new  and  entire  samples, 
respectively.  This  was  caused  by  the  existence  of  the  under-estimation  tendency  in 
the  new  sample  period,  which  was  offset  when  the  entire  sample  period  was 
considered. 


IV.  CONCLUSION 

In  the  work  to  validate  the  reenlistment  equations,  three  central  problems 
were  identified: 

1.  Inability  of  the  equations  to  predict  the  relatively  high 

reenlistment  rates  of  the  1980’s. 

2.  Inability  of  the  equations  to  account  for  the  seemingly 

unsystematic  variations  in  reenlistment  rates  among  some  AFSs. 

3.  Large  variations  in  selected  reenlistment  rates  due  to  small 
numbers  of  observations  per  time  period. 

The  re-specification  of  the  reenlistment  model  directly  addressed  the  first  two  of 
these  problems.  The  problem  of  the  small  number  of  reenlistment  decisions  in  some 
AFSs  and  time  periods  was  addressed  by  extending  15  quarters. 

Re-estimation  of  the  reenlistment  model  equations  involved: 

1.  Extending  the  time  period  used  for  estimation  to  include  fiscal 
years  1982,  1983,  1984,  1985,  and  the  first  three  quarters  of 
FY86. 

2.  Calculating  the  civilian  wage  using  an  age-earning  function 
based  on  age,  sex,  race,  and  education  level  and  adjusted  for 
industry  wages. 

3.  Capturing  the  effect  of  employment  using  a  quadratic  form. 

4.  Adding  seasonality  variables  to  the  equations. 
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5.  Adding  a  variable  to  account  for  the  change  in  the  general  attitude 
of  enlisted  personnel  since  the  end  of  the  Vietnam  Conflict. 

6.  Adding  two  variables  to  account  for  the  ability  of  enlisted 
personnel  to  alter  their  date  of  reenlistment  in  response  to 
pre-announced  changes  in  the  SRB  multiple  for  a  particular 
AFS. 

Each  of  these  changes  in  the  specification  of  the  reenlistment  equations  contributed 
to  improving  their  overall  predictive  capability.  Four  factors  which  were  identified 
from  the  research  were: 

1.  The  persistent  bias  which  was  highly  visible  in  the  validation 
of  the  original  reenlistment  equations  was  reduced  by  the  new 
equations.  Though  a  bias  still  exists  in  the  ability  of  some  of 
the  new  equations  to  predict  over  the  new  sample  period,  the 
bias  did  not  dominate  the  overall  predictive  capability  of  the 
equations. 

2.  The  more  aggregated  the  time  periods,  (i.e.,  fiscal  year  versus 
quarterly)  the  better  the  equations  predicted.  Most 
adjustments  to  AFSs  occur  on  a  monthly  or  quarterly  basis  and, 
thus,  the  more  aggregated  the  time  periods  the  less  the 
internal  personnel  adjustments  were  reflected  in  the  variation 
of  the  reenlistment  rates. 

3.  The  reenlistment  equations  for  AFSs  not  receiving  reenlistment 
bonuses  did  not  predict  as  well  as  for  AFSs  receiving 
reenlistment  bonuses.  The  non-bonus  career '  fields  exhibited 
more  unsystematic  variation  in  reenlistment  rates. 

4.  The  effect  of  reenlistment  bonuses  on  reenlistment  rates  was 
underestimated  unless  the  altered  decision-making  due  to  the 
pre-announcement  of  SRB  multiple  changes  was  accounted  for 
in  the  specification  of  the  equations  and  estimation  of  the 
coefficient  for  bonuses. 


Two  issues  for  future  research  are  suggested  from  the  results  of  this  research: 

1.  Additional  analysis  should  be  performed  on  the  AFSs 
not  receiving  reenlistment  bonuses  in  order  to 
determine  a  better  specification  of  the  model  to 
account  for  the  internal  adjustments  made  where 
manning  overages  have  occurred. 

2.  A  well-defined  set  of  criteria  should  be  developed 
from  the  validation  statistics  presented  in  this 
analysis  to  aid  analysts  in  determining  when  a  model 
is  performing  poorly  enough  to  warrant  re-estimation. 
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APPENDIX  A:  FORECASTING  CREDIBILITY  STATISTICS 


1.1  Goodness  of  Fit  Tests 

One  way  to  test  the  performance  of  the  reenlistment  equations  Is 
to  perform  an  historical  simulation  or  forecast  and  examine  how  closely 
the  dependent  variable  (e.g.,  quarterly  reenlistment  rate)  tracks  its 
corresponding  historical  data  series.  It  is  therefore  desirable  to 
have  some  quantitative  statistical  measure  of  how  closely  the  predicted 
reenlistment  rates  track  their  corresponding  data  series.  Several 
measures  can  be  employed  for  such  tests.  One  measure  that  is  often 
used  is  called  the  Root-Mean-Square  Simulation  Error  (RMSE) . 
(Koutsoyiannis ,  1977)  The  RMSE  for  the  dependent  variable  Y  is 
defined  as : 


RMS  error  - 


(A— 1) 


5 

where  Y£  -  simulated  value  of  the  reenlistment  rate  in  time  period  t 

Yt  -  actual  value  of  the  reenlistment  rate  in  time  period  t 
T  -  number  of  time  periods  in  the  simulation 


The  RMSE  is  a  measure  of  the  deviation  of  the  predicted  reenlistment 
rates  from  their  actual  rates.  Of  course,  the  magnitude  of  this  error 
can  be  evaluated  only  by  comparing  it  with  the  average  size  of  the 
reenlistment  rate.  Another  simulation  error  statistic  is  the  Root- 
Mean-Square  Percent  Error  (RMSPE) ,  which  is  defined  as: 


RMS  percent  error  - 


(A- 2 ) 
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This  is  also  a  measure  of  the  deviation  of  the  predicted  reenlistment 
rates  from  their  actual  values  but  in  percentage  terms.  Two  other 
measures  are  the  mean  simulation  error,  defined  as: 


T 

Mean  error  -  ^  (Y^  -  Y^  )  (A-3) 

t-1 


and  the  mean  percent  error,  defined  as: 


Mean  percent  error  - 


(A-4) 


The  problem  with  mean  errors  is  that  they  may  be  close  to  0  if  large 
positive  errors  cancel  out  large  negative  errors.  The  RMSE  would  be  a 
better  measure  of  the  forecasting  performance.  Mean  absolute  errors 
and  mean  absolute  percent  errors  (MABSE  and  MABSPE)  can  also  be 
calculated  to  avoid  the  problem  of  positive  and  negative  errors 
canceling,  but  RMSE's  are  used  more  often  in  practice,  since  they 
penalize  large  individual  prediction  errors  more  heavily.  The  mean 
errors  are  useful  in  identifying  a  tendency  in  the  prediction  for  over- 
estimation  or  under-estimation. 


Note  that  it  is  entirely  possible  for  an  equation  that  has  a  very 

good  statistical  fit  to  have  a  very  poor  simulation  fit.  In  an 

industry  market  model,  for  example,  an  equation  which  explains  the 

market  price  of  the  good  being  sold  (i.e.,  price  is  the  dependent 

2 

variable)  may  have  a  very  good  statistical  fit  (large  R  ,  small 
standard  errors,  etc).  At  the  same  time,  however,  when  the  model  as  a 
whole  is  simulated,  that  same  price  variable  might  have  a  very  poor 
simulation  fit;  i.e.,  it  might  have  a  very  large  RMSE. 
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Low  RMSE's  are  only  one  desirable  measure  of  a  simulation  fit. 
Another  important  criterion  is  how  well  the  model  simulates  turning 
points  in  the  historical  data.  Consider  Figure  A-l  where  dotted  line  A 
represents  the  historical  time  series  for  some  endogenous  variable  X 
and  solid  lines  B  and  C  represent  the  simulated  values  of  that  same 
variable  using  two  different  models.  From  that  figure  alone,  one  would 
probably  pick  the  model  that  produced  line  C  as  the  better  model,  since 
despite  its  larger  RMSE  it  duplicates  the  market  change  in  variable  X 
that  occurred  historically.  The  model  that  produced  line  B  failed  to 
predict  the  turning  point,  i.e.,  the  sudden  change  in  the  historical 
data.  It  did  track  the  historical  data  closely  during  the  rest  of  the 
simulation  period,  but  any  simple  trend  model  could  have  done  this 
without  really  explaining  the  underlying  physical  processes.  Thus,  the 
ability  of  a  simulation  model  to  duplicate  turning  points  or  rapid 
changes  in  the  actual  data  is  an  important  criterion  for  model 
evaluation. 

A  useful  statistic  related  to  the  RMSE  and  applied  to  the 
evaluation  of  forecasts  is  Theil's  Inequality  Coefficient  (TIC), 
defined  as: 


TIC 


) 


2 


I 

:-l 


+ 


) 


2 


(A— 5 ) 


Note  that  the  numerator  of  TIC  is  just  the  RMSE,  but  the  scaling  of  the 
denominator  is  such  that  TIC  will  always  fall  between  0  and  1.  If  TIC 

-  0,  Y^-  Y^  for  all  t  and  there  is  a  perfect  fit.  If  TIC  -  1,  on  the 
other  hand,  the  predictive  performance  of  the  model  is  as  bad  as  it 
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possibly  could  be.  When  TIC  -  1,  simulated  values  are  always  0  when 
actual  values  are  nonzero,  or  nonzero  predictions  have  been  made  when 

actual  values  are  zero  and  hence  easy  to  predict,  or  simulated  values 

are  positive  (negative)  when  actual  values  are  negative  (positive). 

TIC  can  be  decomposed  in  an  interesting  way.  By  making  a  few 
substitutions,  it  can  be  shown  algebraically  that: 

4"  £  (Yt  "  ^  )2  ”  <?S  “  **  >  +  (ffs"  ffa  )2  +  2(1  "  P)asaa  (A-6) 

where  YS,  Ya,  a  ,  and  a  are  the  means  and  standard  deviations  of  the 

s  a 

series  Yaand  Ya,  respectively,  and  p  is  their  correlation  coefficient. 

We  can  then  define  the  proportions  of  inequality  as: 


TIC*- 


(YS  -  Ya  )2 


(1/T) 


I(Y?- 


(A-7) 


TICS  - 


(»  -  o.) 

s  a 


(l/T) 


I(Y?  - 


(A-8) 


and 


TIC  - 


2(1  -  p)o  o 
s  a 

(1/T)  £( Ya  -  Ya  )2 


(A-9) 
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That  is,  p 
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The  proportions  of  TIC**,  TIC^,  and  TIC^  are  called  the  bias,  the 
variance,  and  the  covariancs  proportions,  respectively,  and  they  are 
useful  as  a  means  of  breaking  the  simulation  error  down  into  its 
characteristic  sources  [  TIC**  +  TICS  +  TICC  -  1  ] . 

The  bias  proportion  TIC11  is  an  indication  of  systematic  error, 
since  it  measures  the  extent  to  which  the  average  values  of  the 
predicted  and  actual  series  deviate  from  each  other.  Whatever  the 
value  of  the  inequality  coefficient  TIC,  one  would  hope  that  TIC**  would 
be  close  to  zero.  A  large  value  TIC11  (above  0.1  or  0.2),  would  be 
quite  troubling,  since  it  would  mean  that  a  systematic  bias  is  present, 
so  that  revision  of  the  model  is  necessary. 

g 

The  variance  proportion  TIC  indicates  the  ability  of  the  model  to 
replicate  the  degree  of  variability  in  the  variable  of  interest.  If 

S 

TIC  Is  large,  It  means  that  the  actual  series  has  fluctuated 
considerably  while  the  simulated  series  shows  little  fluctuation,  or 
vice  versa.  This  would  also  be  troubling  and  might  lead  us  to  a 
revision  of  the  model. 

Finally,  the  covariance  proportion  measures  what  might  be  called 
unsystematic  error;  i.e.,  it  represents  the  remaining  error  after 
deviations  from  average  values  and  average  variabilities  have  been 
accounted  for.  Since  it  is  unreasonable  to  expect  predictions  that  are 
perfectly  correlated  with  actual  outcomes,  this  component  of  error  is 
less  worrisome.  Indeed,  for  any  value  of  TIC  >  0,  the  Ideal 
distribution  of  inequality  over  the  three  sources  is  TIC**  -  TICS  -  0, 
and  TICC  -  1. 


Ll2 _ Additional  Measures  of  Forecastine  Accuracy 

In  addition  to  the  above  standard  measures  of  forecasting 

accuracy,  three  other  measures  will  be  employed:  Janus  Quotient, 
2 

Simulation  R  ,  and  percent  successful  predictions.  The  Janus  Quotient 
provides  a  comparison  between  predictive  capability  in-sample  versus 


out-of-sample  and  Is  defined  as  the  ratio  of  the  average  squared 
differences  out-of-sample  to  the  average  squared  differences  in-sample . 


n  +  m 

I  <Y?  - 

jq  _  -  (A— 10) 

I  (Yt  -  Yt>  2/” 

t  -  1 

where  n  time  periods  are  in-sample  and  m  time  periods  are  out-of- 
sample.  The  Janus  Quotient  ranges  between  zero  and  infinity,  with  a 
value  of  one  indicating  that  the  structure  of  the  model  remains  the 
same  in  the  future  as  in  the  period  of  the  in-sample  estimation.  The 
higher  the  value  of  the  Janus  Quotient,  the  poorer  the  forecasting 
performance  of  the  model.  Values  of  the  Janus  Quotient  higher  than 
unity  are  also  suggestive,  under  certain  conditions,  of  changes  in  the 
structure  of  the  model. 

2 

The  Simulation  R  measures  how  well  the  model  predicts  compared 
to  the  actual  mean  of  the  dependent  variable,  and  is  defined  as, 

I  -  ^>2 

SR  -  1  -  —  -  (A— 11) 

n 

I  -  ^>2 

t  -  1 

where  Y  is  the  actual  out-of-sample  mean  value. 

For  an  out-of-sample  forecast,  the  actual  out-of-sample  mean  value 
is  used  as  a  predictor  of  the  actual  value  for  each  time  period 
compared  with  the  predicted  value  for  each  time  period.  Of  course,  in 
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terms  of  forecasting,  the  analyst  never  knows,  a  priori,  the  value  for 
the  actual  mean  of  the  dependent  variable. 

The  percent  successful  predictions  is  only  useful  for  a  model 
which  predicts  a  dichotomous  dependent  variable  (e.g.  reenlistment  or 
separation)  as  does  a  probit  model.  This  measure  provides  the 
proportion  of  individual  decisions  which  were  correctly  predicted  using 
the  model.  In  the  case  of  a  probit  model,  when  the  predicted 
probability  is  0.5  or  higher,  the  individual  is  assigned  a  one  for  the 
value  of  the  dependent  variable,  e.g,,  the  airman  is  predicted  to 
reenlistment.  If  the  predicted  probability  is  less  than  0.5,  the 
dependent  variable  is  assigned  a  zero,  e.g.,  the  airman  is  predicted  to 
separate . 
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APPENDIX  B:  FIRST  TERM  SIMULATIONS 
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Table  B-l  (Continued) 
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B-2.  First  Term  Simulation  Over  the  1982-1986  Time  Period  by  Quarter  (New  Equations) 
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Table  B-3.  First  Term  Simulation  Over  the  1975-1986  Time  Period  by  Quarter  (New  Equations) 
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